Repository navigation
Add a sample buffer renderer output for AirPlay 2 multi-room - #158
Open
chrizzchrizz wants to merge 1 commit into
Open
chrizzchrizz wants to merge 1 commit into
chrizzchrizz wants to merge 1 commit into
Conversation
New output mode `.sampleBufferRenderer(maxBufferedSeconds:)`: instead of an output audio unit, a timer pulls the engine's manually rendered audio and enqueues it into an AVSampleBufferAudioRenderer driven by an AVSampleBufferRenderSynchronizer. On Mac Catalyst only this output follows the AirPlay devices picked in the system route picker, including several rooms at once (with the long-form audio route sharing policy). The audio is enqueued ahead of what is heard, so progress, duration and the item change callbacks are compensated by the unheard audio content: the previous item is reported until its end is heard, and the callbacks are delivered in order once the change is audible. Seek and item changes flush the renderer, the end of the queue drains, automatic flushes re-enqueue from a history, volume and mute act on the renderer directly. The default output mode stays `.audioUnit`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a sample buffer renderer output for AirPlay 2 multi-room
Problem
On Mac Catalyst the system route picker (
AVRoutePickerView/MPVolumeView) lets the user pick one or more AirPlay 2 devices, for example two rooms at once. When the player uses an output audio unit (AVAudioEngine), picking AirPlay devices does not change the audio session route or the engine configuration, and the output just goes silent. Restarting the engine does not help. Only anAVSampleBufferAudioRendererfollows the devices picked there. Combined with the.longFormAudioroute sharing policy, it also plays to several rooms at once.Changes
A new, opt-in output mode in
AudioPlayerConfiguration:With
.sampleBufferRenderer, no output audio unit is created. Instead, the newSampleBufferRendererOutputuses a timer to pull the engine's rendered audio (AudioPlayerRenderProcessor.renderOutput). It enqueues the audio into anAVSampleBufferAudioRenderer, which anAVSampleBufferRenderSynchronizerdrives.The renderer plays audio enqueued up to
maxBufferedSecondsahead of what is heard, so the player compensates for that latency:progress,durationandframesPlayedreport the position that is heard. The previous item is reported until its end has been heard.didStartPlaying,didFinishPlaying) are delayed until the change can be heard, and they keep their order (OrderedDelayedNotifier).AVSampleBufferAudioRendererWasFlushedAutomatically, e.g. on a route change), the audio is enqueued again from a short history, starting at the flush time.volumeandmutedact on the renderer directly, so they apply at once and no muted audio stays enqueued.The default output mode stays
.audioUnit. Every new code path depends on the renderer output being set, so the existing output is unchanged.Notes
flush(fromSourceTime:)of the renderer always failed in my tests (local output and AirPlay), so a seek uses a full flush. With AirPlay this gives a short rebuffer pause after the seek, which seems to be inherent to AirPlay.Testing
swift test: all tests pass. Builds for macOS, iOS and tvOS..sampleBufferRenderer(maxBufferedSeconds: 4)and the.longFormAudiopolicy: playback on the Mac speakers, on one AirPlay room and on two rooms at once, with seek, pause/resume, item changes and the end of the queue. On iOS the app keeps the default.audioUnitoutput.